Papers with dimensionality reduction
Injecting Wiktionary to improve token-level contextual representations using contrastive learning (2024.eacl-short)
Copied to clipboard
| Challenge: | lexical semantics tasks require contextual word embeddings that are not blind to context, despite the fact that vectors of the same meaning are too different. |
| Approach: | They propose to fine-tune pre-trained language models by using automatically self-augmented examples to target contextual word embeddings. |
| Outcome: | The proposed method achieves significant improvements on the original WiC test set and in two new tests. |
AnnoPlot: Interactive Visualizations of Text Annotations (2024.eacl-demo)
Copied to clipboard
| Challenge: | Annotation projects face challenges in data quality and validity, authors argue . |
| Approach: | They propose an open-source web application that analyzes, manages, and visualizes annotated text data. |
| Outcome: | The proposed application is open-source and promotes transparency and user control . it offers comprehensive views of span annotations and category systems without training or classification model . |
Hard Negative Mining for Domain-Specific Retrieval in Enterprise Systems (2025.acl-industry)
Copied to clipboard
| Challenge: | Existing methods for lexical retrieval struggle due to semantic mismatches and overlapping terminologies, and ambiguous abbreviations common in specialized fields like finance and cloud computing. |
| Approach: | They propose a scalable hard-negative mining framework that dynamically selects semantically challenging but contextually irrelevant documents to enhance deployed re-ranking models. |
| Outcome: | The proposed framework improves on public domain datasets and shows that it is generalizable and ready for real-world applications. |
Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too! (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing topic models rely on probabilistic models to uncover themes within document collections, but are they the only option? |
| Approach: | They propose a way to cluster pre-trained word embeddings while incorporating document information for weighted clustering and reranking top words. |
| Outcome: | The proposed approach performs as well as classical topic models, but with lower runtime and computational complexity. |
DIDS: Domain Impact-aware Data Sampling for Large Language Model Training (2025.emnlp-main)
Copied to clipboard
Weijie Shi, Jipeng Zhang, Yaguang Wu, Jingzhi Fang, Shibo Zhang, Yao Zhao, Hao Chen, Ruiyuan Zhang, Yue Cui, Jia Zhu, Sirui Han, Jiajie Xu, Xiaofang Zhou
| Challenge: | Existing approaches for optimizing domain-level sampling strategies struggle with maintaining intra-domain consistency and accurately measuring domain impact. |
| Approach: | They propose to use a Fisher-Information Matrix-guided metric to measure domain impact to ensure intra-domain consistency and accuracy. |
| Outcome: | The proposed model achieves 3.4% higher average performance while maintaining comparable training efficiency. |
CANarEx: Contextually Aware Narrative Extraction for Semantically Rich Text-as-data Applications (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Narrative modelling is a field of active research that conceptualizes narratives as connected entity chains. |
| Approach: | They propose an alternative narrative extraction approach using semantic role labeling to extract tuples from text, then dimensionality reduction to reduce the space of entities and connections separately. |
| Outcome: | The proposed approach improves on a text-as-data task and improves accuracy and recall. |
Specializing Word Embeddings (for Parsing) by Information Bottleneck (D19-1)
Copied to clipboard
| Challenge: | Pre-trained word embeddings contain rich syntactic and semantic information. |
| Approach: | They propose a variational information bottleneck method to nonlinearly compress word embeddings . they compress each word embedded to either a discrete tag or a continuous vector . |
| Outcome: | The proposed method can be extended to other syntactic or semantic tasks. |
Bridging Fairness and Environmental Sustainability in Natural Language Processing (2022.emnlp-main)
Copied to clipboard
| Challenge: | a lack of research on the interplay between fairness and environmental impact is a problem in natural language processing . fairness is prone to encode and amplify stereotypical social biases, according to several studies . |
| Approach: | They evaluate a technique to reduce energy consumption of English NLP models by knowledge distillation for its impact on fairness. |
| Outcome: | The proposed method reduces energy consumption and environmental impact of English NLP models. |
Sentence Meta-Embeddings for Unsupervised Semantic Textual Similarity (2020.acl-main)
Copied to clipboard
| Challenge: | Existing word embeddings combine complementary strengths of their components to achieve unsupervised semantic similarity (STS). |
| Approach: | They propose to ensemble pre-trained sentence encoders into sentence meta-embeddings to achieve unsupervised Semantic Textual Similarity (STS) they adapt dimensionality reduction, generalized Canonical Correlation Analysis and cross-view auto-encoders to their work. |
| Outcome: | The proposed method achieves 3.7% to 6.4% Pearson’s r over single-source word embeddings on the STS Benchmark and on the StS12-STS16 datasets. |
Mapping the Circumplex of Affect: Geometric Analysis of Emotion Representations via Hyperspherical Contrastive Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to induce circular emotion representations in language models are limited . elucidates trade-offs involved in applying circumplex models to deep learning architectures . |
| Approach: | They propose a method to induce circular emotion representations within language models via contrastive learning on a hypersphere. |
| Outcome: | The proposed method underperforms in high-dimensional settings and fine-grained classification. |
SMEC:Rethinking Matryoshka Representation Learning for Retrieval Embedding Compression (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models generate high-dimensional embeddings that capture rich semantic and syntactic information. |
| Approach: | They propose a training framework to reduce dimensionality and complexity of large language models. |
| Outcome: | Experiments on image, text, and multimodal datasets show that the proposed training framework reduces dimensionality while maintaining performance. |